Simple macro benchmark #2192
Closed
Simple macro benchmark #2192
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2192 +/- ##
==========================================
- Coverage 72.77% 67.66% -5.11%
==========================================
Files 89 101 +12
Lines 7676 8273 +597
==========================================
+ Hits 5586 5598 +12
- Misses 2090 2675 +585
Flags with carried forward coverage won't be shown. Click here to find out more.
|
zregvart
force-pushed
the
issue/EC-968
branch
2 times, most recently
from
965e4a4 to
8892a99
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
zregvart
force-pushed
the
issue/EC-968
branch
from
8892a99 to
e97a137
Compare
zregvart
force-pushed
the
issue/EC-968
branch
2 times, most recently
from
5de07b4 to
ca57b86
Compare
|
I ran the benchmark on main (this branch) and with reverted #2184, this gives us some data to work with: |
|
Or this: Gives:
|
zregvart
force-pushed
the
issue/EC-968
branch
from
ca57b86 to
8a9497d
Compare
zregvart
force-pushed
the
issue/EC-968
branch
from
8a9497d to
6710bd1
Compare
Implementation of a simple macro benchmark executing `ec validate image` with the fixed state and no external dependencies. For this the OCI distribution registry is run in a container with the data from the `benchmark/simple/data.tar.gz` that contains the copy of all images/blobs that the command and the policy rules currently access and the git repository. The benchmark outputs in the golang standard benchmark format that can be utilized with tools in the golang benchmarking ecosystem. Due to the global `downloadCache` variable in `internal/policy/source/source.go` the benchmark cannot be run in parallel. So only invoking the benchmark once is currently possible. Reference: https://issues.redhat.com/browse/EC-968
zregvart
force-pushed
the
issue/EC-968
branch
from
6710bd1 to
ac18ba7
Compare
|
I'll give it a quick rebase to fix the go.mod conflict, then I suggest merging. Lgtm. |
|
Not sure if the codecov results are accurate, but either way, not a blocker for this PR. Will merge soon if no objections. |
|
I tried to run it locally and hit this: which I think is the same problem as https://issues.redhat.com/browse/EC-1041 . |
|
I'm hitting a different problem now trying to run this locally, so not confident about merging. Because I'm adding extra commits, I'll redo the PR with my fork, see #2304. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implementation of a simple macro benchmark executing
ec validate imagewith the fixed state and no external dependencies.For this the OCI distribution registry is run in a container with the data from the
benchmark/simple/data.tar.gzthat contains the copy of all images/blobs that the command and the policy rules currently access and the git repository.The benchmark outputs in the golang standard benchmark format that can be utilized with tools in the golang benchmarking ecosystem.
Due to the global
downloadCachevariable ininternal/policy/source/source.gothe benchmark cannot be run in parallel. So only invoking the benchmark once is currently possible.Reference: https://issues.redhat.com/browse/EC-968